Goto

Collaborating Authors

 computer system research


The Unreasonable Ineffectiveness of Machine Learning in Computer Systems Research

#artificialintelligence

In 1960, the physicist Eugene Wigner wrote a famous essay titled "The Unreasonable Effectiveness of Mathematics in the Natural Sciences" in which he explored the question of why mathematics is so remarkably useful in the natural sciences. A contemporary example of such "unreasonable effectiveness" is the success that machine learning has had in transforming many disciplines in the past decade. Particularly impressive is the progress in autonomous vehicles. In the 2004 DARPA Grand Challenge for autonomous vehicles, which popularized the idea of driverless cars, none of the vehicles was able to complete a relatively simple route through the Mojave Desert, and I thought it unlikely that I would see driverless cars operating in urban environments in my lifetime. Since that time, progress in this area has been phenomenal, thanks to rapid advances in using machine learning for sensing and navigation (and in building low-cost sensors and controls).


Repeatability in Computer Systems Research

Communications of the ACM

In 2012, when reading a paper from a recent premier computer security conference, we came to believe there is a clever way to defeat the analyses asserted in the paper, and, in order to show this we wrote to the authors (faculty and graduate students in a highly ranked U.S. computer science department) asking for access to their prototype system. We thus decided to reimplement the algorithms in the paper but soon encountered obstacles, including a variable used but not defined; a function defined but never used; and a mathematical formula that did not typecheck. We asked the authors for clarification and received a single response: "I unfortunately have few recollections of the work ... " We next made a formal request to the university for the source code under the broad Open Records Act (ORA) of the authors' home state. The university's legal department responded with: "We have been unable to locate a confirmed instance of [system's] source code on any [university] system."